Search Results: "tino"

25 August 2015

Lunar: Reproducible builds: week 17 in Stretch cycle

A good amount of the Debian reproducible builds team had the chance to enjoy face-to-face interactions during DebConf15.
Names in red and blue were all present at DebConf15
Picture of the reproducible builds talk during DebConf15
Hugging people with whom one has been working tirelessly for months gives a lot of warm-fuzzy feelings. Several recorded and hallway discussions paved the way to solve the remaining issues to get reproducible builds part of Debian proper. Both talks from the Debian Project Leader and the release team mentioned the effort as important for the future of Debian. A forty-five minutes talk presented the state of the reproducible builds effort. It was then followed by an hour long roundtable to discuss current blockers regarding dpkg, .buildinfo and their integration in the archive. Picture of the reproducible builds roundtable during DebConf15 Toolchain fixes Reiner Herrmann submitted a patch to make rdfind sort the processed files before doing any operation. Chris Lamb proposed a new patch for wheel implementing support for SOURCE_DATE_EPOCH instead of the custom WHEEL_FORCE_TIMESTAMP. akira sent one making man2html SOURCE_DATE_EPOCH aware. St phane Glondu reported that dpkg-source would not respect tarball permissions when unpacking under a umask of 002. After hours of iterative testing during the DebConf workshop, Sandro Knau created a test case showing how pdflatex output can be non-deterministic with some PNG files. Packages fixed The following 65 packages became reproducible due to changes in their build dependencies: alacarte, arbtt, bullet, ccfits, commons-daemon, crack-attack, d-conf, ejabberd-contrib, erlang-bear, erlang-cherly, erlang-cowlib, erlang-folsom, erlang-goldrush, erlang-ibrowse, erlang-jiffy, erlang-lager, erlang-lhttpc, erlang-meck, erlang-p1-cache-tab, erlang-p1-iconv, erlang-p1-logger, erlang-p1-mysql, erlang-p1-pam, erlang-p1-pgsql, erlang-p1-sip, erlang-p1-stringprep, erlang-p1-stun, erlang-p1-tls, erlang-p1-utils, erlang-p1-xml, erlang-p1-yaml, erlang-p1-zlib, erlang-ranch, erlang-redis-client, erlang-uuid, freecontact, givaro, glade, gnome-shell, gupnp, gvfs, htseq, jags, jana, knot, libconfig, libkolab, libmatio, libvsqlitepp, mpmath, octave-zenity, openigtlink, paman, pisa, pynifti, qof, ruby-blankslate, ruby-xml-simple, timingframework, trace-cmd, tsung, wings3d, xdg-user-dirs, xz-utils, zpspell. The following packages became reproducible after getting fixed: Uploads that might have fixed reproducibility issues: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: St phane Glondu reported two issues regarding embedded build date in omake and cduce. Aur lien Jarno submitted a fix for the breakage of make-dfsg test suite. As binutils now creates deterministic libraries by default, Aur lien's patch makes use of a wrapper to give the U flag to ar. Reiner Herrmann reported an issue with pound which embeds random dhparams in its code during the build. Better solutions are yet to be found. reproducible.debian.net Package pages on reproducible.debian.net now have a new layout improving readability designed by Mattia Rizzolo, h01ger, and Ulrike. The navigation is now on the left as vertical space is more valuable nowadays. armhf is now enabled on all pages except the dashboard. Actual tests on armhf are expected to start shortly. (Mattia Rizzolo, h01ger) The limit on how many packages people can schedule using the reschedule script on Alioth has been bumped to 200. (h01ger) mod_rewrite is now used instead of JavaScript for the form in the dashboard. (h01ger) Following the rename of the software, debbindiff has mostly been replaced by either diffoscope or differences in generated HTML and IRC notification output. Connections to UDD have been made more robust. (Mattia Rizzolo) diffoscope development diffoscope version 31 was released on August 21st. This version improves fuzzy-matching by using the tlsh algorithm instead of ssdeep. New command line options are available: --max-diff-input-lines and --max-diff-block-lines to override limits on diff input and output (Reiner Herrmann), --debugger to dump the user into pdb in case of crashes (Mattia Rizzolo). jar archives should now be detected properly (Reiner Herrman). Several general code cleanups were also done by Chris Lamb. strip-nondeterminism development Andrew Ayer released strip-nondeterminism version 0.010-1. Java properties file in jar should now be detected more accurately. A missing dependency spotted by St phane Glondu has been added. Testing directory ordering issues: disorderfs During the reproducible builds workshop at DebConf, participants identified that we were still short of a good way to test variations on filesystem behaviors (e.g. file ordering or disk usage). Andrew Ayer took a couple of hours to create disorderfs. Based on FUSE, disorderfs in an overlay filesystem that will mount the content of a directory at another location. For this first version, it will make the order in which files appear in a directory random. Documentation update Dhole documented how to implement support for SOURCE_DATE_EPOCH in Python, bash, Makefiles, CMake, and C. Chris Lamb started to convert the wiki page describing SOURCE_DATE_EPOCH into a Freedesktop-like specification in the hope that it will convince more upstream to adopt it. Package reviews 44 reviews have been removed, 192 added and 77 updated this week. New issues identified this week: locale_dependent_order_in_devlibs_depends, randomness_in_ocaml_startup_files, randomness_in_ocaml_packed_libraries, randomness_in_ocaml_custom_executables, undeterministic_symlinking_by_rdfind, random_build_path_by_golang_compiler, and images_in_pdf_generated_by_latex. 117 new FTBFS bugs have been reported by Chris Lamb, Chris West (Faux), and Niko Tyni. Misc. Some reproducibility issues might face us very late. Chris Lamb noticed that the test suite for python-pykmip was now failing because its test certificates have expired. Let's hope no packages are hiding a certificate valid for 10 years somewhere in their source! Pictures courtesy and copyright of Debian's own paparazzi: Aigars Mahinovs.

16 August 2015

Lunar: Reproducible builds: week 16 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Valentin Lorentz sent a patch for ispell to initialize memory structures before dumping their content. In our experimental repository, qt4-x11 has been rebased on the latest version (Dhole), as was doxygen (akira). Packages fixed The following packages became reproducible due to changes in their build dependencies: backup-manager, cheese, coinor-csdp, coinor-dylp, ebook-speaker, freefem, indent, libjbcrypt-java, qtquick1-opensource-src, ruby-coffee-script, ruby-distribution, schroot, twittering-mode. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: akira found another embedded code copy of texi2html in maxima. reproducible.debian.net Work on testing several architectures has continued. (Mattia/h01ger) Package reviews 29 reviews have been removed, 187 added and 34 updated this week. 172 new FTBFS reports were filled, 137 solely by Chris West (Faux). josch spent time investigating the issue with fonts in PDF files. Chris Lamb documented the issue affecting documentation generated by ocamldoc. Misc. Lunar presented a general Reproducible builds HOWTO talk at the Chaos Communication Camp 2015 in Germany on August 13th. Recordings are already available, as well as slides and script. h01ger and Lunar also used CCCamp15 as an opportunity to have discussions with members of several different projects about reproducible builds. Good news should be coming soon.

5 August 2015

Antonio Terceiro: Elixir in Debian, MiniDebconf at FISL, and Debian CI updates

In June I started keeping track of my Debian activities, and this is my July update. Elixir in Debian Elixir is a functional language built on top of the Erlang virtual machine. If features imutable data structures, interesting concurrency primitives, and everything else that Erlang does, but with a syntax inspired by Ruby what makes it much more aproachable in my opinion. Those interested in Elixir for Debian are encouraged to hang around in #debian-elixir on the OFTC IRC servers. There are still a lot of things to figure out, for example how packaging Elixir libraries and applications is going to work. MiniDebconf at FISL, and beyond I helped organize a MiniDebconf at this year s FISL, in Porto Alegre on the 10th of July. The whole program was targetted at getting more people to participate in Debian, so there were talks about translation, packaging, and a few other more specific topics. I myself gave two talks: one about Debian basics, What is Debian, and how it works , and second one on packaging the free software web , which I will also give at Debconf15 later this month. The recordings are available (all talks in Portuguese) at the Debian video archive thanks to Holger Levsen. We are also organizing a new MiniDebconf in October as part of the Latinoware schedule. Ruby We are in the middle of a transition to switch to Ruby 2.2 as default in Debian unstable, and we are almost there. The Ruby transition is now on hold while GCC 5 one is going on, but will be picked up as soon as were are done with GCC 5. ruby-defaults has been uploaded to experimental for those that want to try having Ruby 2.2 as default before that change hits unstable. I myself have been using Ruby 2.2 as default for several weeks without any problem so far, including using vagrant on a daily basis and doing all my development on sid with it. I started taking notes about Ruby interpreter transitions work to make sure that knowledge is registered. I have uploaded minor security updates of both ruby2.1 and ruby2.2 to unstable. They both reached testing earlier today. I have also fixed another bug in redmine, which I hope to get into stable as well as soon as possible. gem2deb has seen several improvements through versions 0.19, 0.20, 0.20.1 and 0.20.2. I have updated a few packages: Two NEW packages, ruby-rack-contrib and ruby-grape-logging ,were ACCEPTED into the Debian archive. Kudos to the ftp-master team who are doing an awesome job reviewing new packages really fast. Debian Continuous Integration This month I have made good progress with the changes needed to make debci work as a distributed system with one master/scheduler node and as many worker nodes (running tests) as possible. While doing my tests, I have submitted a patch to lxc and updated autodep8 in unstable. At some point I plan to upload both autodep8 and autopkgtest to jessie-backports. Sponsoring I have sponsored a few packages:

10 June 2015

Norbert Preining: Tribute to Hermann Zapf

Last weekend Hermann Zapf, one of the greatest font designers and typographers of our times, has passed away at the age of 96. Much has been written about his life, his immense sphere of influence, his excellent typefaces, and some of the links to other obituaries can be found here at the end. There is nothing of value I can contribute, but let Hermann Zapf speak for himself with a few images from two books by or dedicated to him. zapf-heart-blood-ink The books I have taken these photos from are: zapf-books In his long life, Hermann Zapf has created a wide range of typefaces, some if which have become the main staple in the printing business, most notably Palatino, Janson, Optima, and Zapf Dingbats. But there are many more: zapf-alphabets While probably not as popular anymore as some years (decades?) ago, Palatino still is one of my favorite typefaces. I often wished to have the freedom to choose my own fonts for publications, but normally that doesn t happen. Palatino has found many offspring (or imitations?), and there are excellent renderings for use with (La)TeX, too. zapf-palatino Hermann Zapf also had connections with Don Knuth and the AMS (American Mathematical Society), and he designed for the AMS the Euler typeface, using Knuth s MetaFont program. zapf-euler Last weekend, when I heard of Hermann Zapf s death, I was at a conference at Tsukuba University. What a coincidence that all the signs in the building (floor numbers, indications, etc) were set optimain another of his great typefaces, the Optima. I had a strange feeling, and reset my presentation in Optima/Euler in honor of Hermann Zapf s great life. Although I never met Hermann Zapf personally I missed the TeX conferences where he participated it was to a great extent his influence that years ago I got started in typography. Living a life like he did, dedicating himself to the beauty and expressiveness of letters, design, calligraphy, seems like a dream for me. But beauty can be found even in the strangest equation in mathematics, the queerest theorem in logic. While not all of us are as gifted as Hermann Zapf in creating beauty, we should strive to increase our ability to appreciate the beauty. The beauty of letters is subtle, evasive, and needs training, that is practice, to be appreciated. Hermann Zapf taught us how even self-training can lead to excellence. His letters, which I meet every day, are a permanent reminder to strive for beauty and clarity, and never relent in our exercise. Thanks Hermann Zapf. zapf-letters Links to tributes around the world English Linotype (Type foundry where Zapf started working)
Bigelow and Holmes (Type designers)
New York Times (Newspaper)
Type Drawers
Quarz (News magazine)
Kaveh Bazargan (next president of TUG)
TeX Overflow (Q&A website concerning TeX) German FontShop (Type foundry and internet shop)
Deutschland Funk (News station)
S ddeutsche (Newspaper)
Echo (Newspaper)
Heise (Techology News page)

17 May 2015

Lunar: Reproducible builds: week 3 in Stretch cycle

What happened about the reproducible builds effort for this week: Toolchain fixes Tomasz Buchert submitted a patch to fix the currently overzealous package-contains-timestamped-gzip warning. Daniel Kahn Gillmor identified #588746 as a source of unreproducibility for packages using python-support. Packages fixed The following 57 packages became reproducible due to changes in their build dependencies: antlr-maven-plugin, aspectj-maven-plugin, build-helper-maven-plugin, clirr-maven-plugin, clojure-maven-plugin, cobertura-maven-plugin, coinor-ipopt, disruptor, doxia-maven-plugin, exec-maven-plugin, gcc-arm-none-eabi, greekocr4gamera, haskell-swish, jarjar-maven-plugin, javacc-maven-plugin, jetty8, latexml, libcgi-application-perl, libnet-ssleay-perl, libtest-yaml-valid-perl, libwiki-toolkit-perl, libwww-csrf-perl, mate-menu, maven-antrun-extended-plugin, maven-antrun-plugin, maven-archiver, maven-bundle-plugin, maven-clean-plugin, maven-compiler-plugin, maven-ear-plugin, maven-install-plugin, maven-invoker-plugin, maven-jar-plugin, maven-javadoc-plugin, maven-processor-plugin, maven-project-info-reports-plugin, maven-replacer-plugin, maven-resources-plugin, maven-shade-plugin, maven-site-plugin, maven-source-plugin, maven-stapler-plugin, modello-maven-plugin1.4, modello-maven-plugin, munge-maven-plugin, ocaml-bitstring, ocr4gamera, plexus-maven-plugin, properties-maven-plugin, ruby-magic, ruby-mocha, sisu-maven-plugin, syncache, vdk2, wvstreams, xml-maven-plugin, xmlbeans-maven-plugin. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Ben Hutchings also improved and merged several changes submitted by Lunar to linux. Currently untested because in contrib: reproducible.debian.net
Thanks to the reproducible-build team for running a buildd from hell. gregor herrmann
Mattia Rizzolo modified the script added last week to reschedule a package from Alioth, a reason can now be optionally specified. Holger Levsen splitted the package sets page so each set now has its own page. He also added new sets for Java packages, Haskell packages, Ruby packages, debian-installer packages, Go packages, and OCaml packages. Reiner Herrmann added locales-all to the set of packages installed in the build environment as its needed to properly identify variations due to the current locale. Holger Levsen improved the scheduling so new uploads get tested sooner. He also changed the .json output that is used by tracker.debian.org to lists FTBFS issues again but only for issues unrelated to the toolchain or our test setup. Amongst many other small fixes and additions, the graph colors should now be more friendly to red-colorblind people. The fix for pbuilder given in #677666 by Tim Landscheidt is now used. This fixed several FTBFS for OCaml packages. Work on rebuilding with different CPU has continued, a kvm-on-kvm build host has been set been set up for this purpose. debbindiff development Version 19 of debbindiff included a fix for a regression when handling info files. Version 20 fixes a bug when diffing files with many differences toward a last line with no newlines. It also now uses the proper encoding when writing the text output to a pipe, and detects info files better. Documentation update Thanks to Santiago Vila, the unneeded -depth option used with find when fixing mtimes has been removed from the examples. Package reviews 113 obsolete reviews have been removed this week while 77 has been added.

9 May 2015

Daniel Silverstone: Kimchi Trial 1 - Part 1

I have spent today making my first ever batch of Kimchi. I have been documenting it in photos as I go, but thought I'd write up what I did so that if anyone else fancies having a go too, we can compare results. For a start, this recipe is nowhere near "traditional" because I don't have access to certain ingredients such as glutinous rice flour. I'm sure if I searched in many of the asian supermarkets around the city centre I could find it, but I'm lazy so I didn't even try. I am not writing this up as a traditional recipe because I'm kinda making it up as I go along, with hints from various sources including the great and wonderful Maangchi whose YouTube channel I follow. Observant readers or followers of Maangchi will recognise the recipe as being close to her Easy Kimchi recipe, however since I'm useless, it won't be exact. If this batch turns out okay then I'll write it up as a proper recipe for you to follow. I started off with three Chinese Leaf cabbages which seemed to be about 1.5kg or so once I'd stripped the less nice outer leaves, cored and chopped them. Chopped up cabbage I then soaked and drained the cabbage in cold water... Soaking cabbage ...before sprinkling a total of one third of a cup of salt over the cabbage and mixing it to distribute the salt. Salted Cabbage Then I returned to the cabbage every 30 minutes to re-mix it a total of three times. After the cabbage had been salted for perhaps 1h45m or so, I rinsed it out. Maangchi recommends washing the cabbage three times so that's what I did before setting it out to drain in a colander. Drained salted cabbage 1h45m later Maangchi then calls for the creation of a porridge made from sweet rice flour which it turns out is very glutinous. Since I lack the ability to get that flour easily I substituted cornflour which I hope will be okay and then continued as before. One cup of water, one third of a cup of cornflour was heated until it started to bubble and then one sixth of a cup of sugar was added. Stirred throughout, once it went translucent I turned the heat off and proceeded. Porridge thingy One half of a red onion, a good thumb (once peeled) of ginger, half a bulb of garlic and one third of a cup of fish sauce went into a mini-zizzer. I then diagonal-chopped about five spring onions, and one leek, before cutting a fair sized carrot into inch long pieces before halving and then thinly slicing it. Maangchi calls for julienned carrots but I am not that patient. Veggybits Into the cooled porridge I put two thirds of a cup of korean hot pepper flakes (I have the coarse, but a mix of coarse and fine would possibly be better), the zizzed onion/garlic/ginger/fish mix and the vegetables... All in the pan ...before mixing that thoroughly with a spatula. Mixed vegetables Next came the messy bit (I put latex gloves on, and thoroughly washed my gloved hands for this). Into my largest mixing bowl I put a handful of the drained cabbage into the bowl and a handful of the pepper mix. Thoroughly mixing this before adding another handful of cabbage and pepper mix I repeated until all the cabbage and hot pepper mixed vegetables are well combined. I really got your arms into it, squishing it around, separating the leek somewhat, etc. Bowl of kimchi As a final task, I scooped the kimchi into a clicklok type box, pressing it down firmly to try and remove all the air bubbles, before sealing it in for a jolly good fermenting effort. I will have to remove a little tonight for our dinner (beef strips marinated in onion, ginger and garlic, with kimchi on rice) but the rest will then sit to ferment for a bit. Expect a part-2 with the report from tonight's dinner and a part-3 with the results after fermentation. Box of kimchi
As an aside, I got my hot pepper flakes from Sous Chef who, it turns out, also stock glutinous rice flour -- I may have to get some more from them in the future. (#notsponsored)

6 May 2015

Matthieu Caneill: Debsources got swag and continous integration

Debsources (http://sources.debian.net) is still under active development. We recently had a Gnome Outreachy intern, Jingjie Jiang, and we're about to work with 2 GSoC students, Cl ment Schreiner and Orestis Ioannou. I will present here the GitHub mirror we've set up, in order to allow external pull requests to be submitted, and to use the continous integration service provided by Travis-CI. GitHub and Travis-CI Debsources' source code is hosted on Debian's git servers, and from there is mirrored to GitHub. Every time a commit is pushed (to master or other branches) or a pull request is open, the test suite will be automatically run on Travis-CI, and the result (tests pass or don't) is displayed on GitHub. This allows us to quickly filter external contributions (when they are submitted on GitHub), and be sure everything works with our setup, before reviewing work. Travis-CI runs the tests on OpenVZ containers. The complete infrastructure was a bit challenging to setup, but as we now have a Docker recipe to quicly begin to hack on Debsources, most of the work could be done using the Dockerfile instructions. In average, a run on Travis-CI (which includes git cloning the code and test data, setup the server, and run the tests suite) takes 7 minutes, which is an ok amount of time to wait for before submitting a pull request, in my opinion. Bugs discovered in the process Setting up this continuous integration infrastructure made me discover a few bugs. Python magic does black magic Debsources runs fine on Debian (not surprisingly), but I got tricked by black magic when I tried to run it on Ubuntu (which is the OS run in Travis-CI's containers). We use the magic library to guess the type of files we're dealing with, for instance when we need to decide between rendering a file (for text files) or downloading it (for binary files). Here comes the tricky part: the Python bindings for libmagic are not the same in Debian and Pypi. Debsources uses Debian package python-magic, which is not in Ubuntu 12.04. Moreover, there's no Python egg for it on Pypi, which has however another package (called magic) which provides a different API. I solved this with a dirty hack, using the fact python-magic lies in a single file: 
mkdir /tmp/python-magic && wget https://raw.githubusercontent.com/file/file/master/python/magic.py -O /tmp/python-magic/magic.py && export PYTHONPATH=/tmp/python-magic/:$PYTHONPATH
It simply downloads the library, saves it in a temporary folder and includes it in the Python path. Let's see for how long it works before everything breaks! Size of a directory One test in the suite was ensuring the information returned by ls -l on a directory and stored in the DB was the right information. Inode metadata was tested, such as name, permissions, type, or size. Interestingly enough, the size of a directory was tested, and expected to be 4096 bytes. The size of a directory actually depends on the filesystem in use, and on the number of files this directory contains. We often see 4096 because it's the size of a not-too-big directory on ext4. Travis-CI doesn't use ext4: 
$ df -T
Filesystem            Type     1K-blocks      Used Available Use%
Mounted on
/vz/private/209140041 simfs    125829120 103460612  22368508  83% /
none                  devtmpfs   1572864         8   1572856   1% /dev
none                  tmpfs       314576        56    314520   1% /run
none                  tmpfs         5120         4      5116   1%
/run/lock
none                  tmpfs      1572864         0   1572864   0%
/run/shm
/dev/null             tmpfs       786432    171584    614848  22%
/var/ramfs
Simfs is a container filesystem for OpenVZ, on which directories have different sizes than on ext4: 
$ ls -al /
total 0
drwxr-xr-x 23 root     root      480 Feb  4 18:08 .
drwxr-xr-x 23 root     root      480 Feb  4 18:08 ..
drwxr-xr-x  2 root     root     2480 Feb  4 18:20 bin
drwxr-xr-x  2 root     root       40 Apr 19  2012 boot
drwxr-xr-x  5 root     root      660 Apr 30 13:56 dev
drwxr-xr-x 99 root     root     3560 Apr 30 13:56 etc
-rw-r--r--  1 root     root        0 Feb  4 17:56 fastboot
drwxr-xr-x  3 root     root       80 Feb  4 17:57 home
[...]
Directory sizes are not even powers of 2. Hence I changed the test to not check directory sizes. Hopefully this will help to make Debsources work on more filesystems! An empty file is hiding Last but not least, because this bug is still open in the wild. A file, which appears to be empty, is not taken into account by Debsources' updater. This file is sources/non-free/m/make-doc-non-dfsg/4.0-2/.pc/applied-patches. It is present in the filesystem in the container, is not the only empty file over there, but still doesn't appear in the database, and make fail the test which counts files. The test has been commented out (booooooh), so that we still can use Travis-CI's platform for our GSoC students, before it's fixed. Conclusion Making Debsources run automatically on a different platform as the one we usually use permitted us to spot bugs, write dirty hacks, and expand the filesystems it's supposed to run on. Now, let's hope the continuous integration will help our GSoC students, and let's wish them good luck!

31 March 2015

Konstantinos Margaritis: "Advanced Java EE Development with WildFly" released by Packt (I was one of the reviewers!)

For the past months I had the honour and pleasure of being one of the reviewers of "Advanced Java EE Development with WildFly" by Deepak Vohra. Today, I'm pleased to announce that the book has just been released by Packt: https://www.packtpub.com/application-development/advanced-java-ee-development-wildfly It was my first time being a reviewer and it was a very interesting experience. I would like to thank the two Project Coordinators from Packt, Aboli Ambardekar and Suzanne Coutinho, who guided me with the reviewing process, so that my review would be as accurate as possible and only related to technical aspect of the book. Looking at the process retrospectively I now begin to understand the complication of achieving a balance between the author's vision for the book and the scrutiny of the (many) reviewers. And of course I would like to thank the author, Deepak Vohra, for writing the book in the first place, I'm looking forward to reading the actual physical book :)

29 November 2014

Dirk Eddelbuettel: RcppArmadillo 0.4.550.1.0

A week ago, Conrad provided another minor release 4.550.0 of Armadillo which has since received one minor correction in 4.550.1.0. As before, I had created a GitHub-only pre-release of his pre-release which was tested against the almost one hundred CRAN dependents of our RcppArmadillo package. This passed fine as usual, and results are as always in the rcpp-logs repository. Processing and acceptance at the CRAN took a little longer as around the same time a fresh failure in unit tests had become apparent on an as-of-yet unannounced new architecture (!!) also tested at CRAN. The R-devel release has since gotten a new capabilities() test for long double, and we now only run this test (for our rmultinom()) if the test asserts that the given R build has this capability. Phew, so with all that the new version in now on CRAN; Windows binaries have been built and I also uploaded new Debian binaries. Changes are summarized below; our end also includes added support for conversion of Field types takes to short pull request by Romain.
Changes in RcppArmadillo version 0.4.550.1.0 (2014-11-26)
  • Upgraded to Armadillo release Version 4.550.1 ("Singapore Sling Deluxe")
    • added matrix exponential function: expmat()
    • faster .log_p() and .avg_log_p() functions in the gmm_diag class when compiling with OpenMP enabled
    • faster handling of in-place addition/subtraction of expressions with an outer product
    • applied correction to gmm_diag relative to the 4.550 release
  • The Armadillo Field type is now converted in as<> conversions
Courtesy of CRANberries, there is also a diffstat report for the most recent release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

10 November 2014

Konstantinos Margaritis: New owner/maintainer for CSVChart Drupal module

Back in ~2008, I had created a small Drupal module that used CustomFilter and Google Chart Tools: Image Charts, called CSV Chart. It was a simple module that took embedded CSV data and presented them as Google charts. It was very handy for what I wanted -and still want- to do, present benchmarks for my work on Altivec (and now in general SIMD). However, I don't really code in PHP anymore -haven't coded PHP since 2010- so the module was left to bitrot. I did some minor adjustment to run with the current D7-based site, but that was it. Thankfully, others found the module useful to use and adopt and the result has been that in the last days I transfered ownership of the CSV Chart to Pierre Vriens, so that he can continue development and maintenance of this tiny module. I would like to publicly thank Pierre for his work! And that's the beauty of Free Software!

31 October 2014

Konstantinos Margaritis: SIMD on javascript, MHO

Just read the Mozilla announcement on SIMD.js and I can say I got mixed feelings about this. I don't really comment other news/blogs/announcements, but this is an exception. On one hand, I definitely welcome more SIMD use everywhere, being a SIMD advocate and enthusiast for many years (since 2004 actually). So seeing more of it and by someone such as Mozilla, that's even better! On the other hand, wait, that's SIMD in Javascript?!? Really? Why? As if we already covered native coverage of SIMD in every other part, including the browser itself (No browser uses SIMD extensively in its core, though that would prove to be of actual benefit, the only SIMD code I know of is in the media playback code, which is usually some external library like ffmpeg/x264/etc that already has SIMD optimized parts anyway. So, instead of using resources to optimize the core browser with SIMD -I'm sure there will be plenty of opportunities in the codebase for such optimizations- so that every web application, including Javascript ones will be optimized, or even the JS JITs themselves, yet Mozilla wants to push the effort to the web app developer to use their SIMD.js to do the equivalent of what SIMD coders have been doing to native apps for a long time now, only for JS apps. Ok, so what's the gain? I read the PDF presentation that shows mandebrot.js to go from 9 FPS to 37FPS using SIMD.js. Admittedly that's impressive. But it also proves that the whole buzz about lower energy footprint computers, power efficiency is just useless. Why is that? For comparison running Xaos (fractal/mandelbrot program) on my very low end PC (2-core Athlon X2, AM2 socket, so DDR2) gives me ~250FPS, and I'm not even sure it's using SSE at all (from a simple check it doesn't). Zooming is realtime and at full detail. In the same talk, there was a benchmark of LLVM Javascript being as fast as C++ or 1.5x the native running time. I admit haven't tried the tests listed, but the mandelbrot test was using asm.js and 9 is definetely not 250/1.5. But I guess I'm just picky. So, the latest trend of moving everything to the browser and JS,means that instead of optimizing my apps to run great on native, instead of making stuff running faster on my 5Wt big.Little 8-core ARM SoC, I have to get a much more power-hungry CPU to see the same performance. I'm totally against that. I want my newer CPUs, who are more energy-efficient and faster to actually feel like THAT. I don't want to upgrade just to experience the performance of a 486 20 years ago! The talk mentioned HTML5 (and hence javascript) overtaking all other platforms for application development everywhere, including the smartphones. I certainly hope that's not the case, and I know of many people who also don't feel that way. We're not buying the "Everything on the web/cloud" paradigm, but I guess we're just a minority. I could go on for a long time, but I have an actual SIMD-related bug to fix, cheers. Note: I used to have comments enabled on my blog, but moderating spam was too time consuming, even with CAPTCHAs, so I disabled them entirely, if anyone could suggest of a better method, I'd gladly take advice -have been thinking about disqus, not sure if it's actually a good solution).

22 October 2014

Konstantinos Margaritis: Eigen NEON port extended to ARMv8!

Soon after the VSX port, and as promised I have completed the ARMv8 NEON (a.k.a. Advanced SIMD) port. Basically this extends support to 64-bit doubles and also provides faster alternatives to division as ARMv8 has builtin instructions for division both for 32-bit floats and 64-bit doubles. Preliminary benchmarks (bench_gemm):

13 October 2014

Konstantinos Margaritis: SIMD optimizations, cont.

A friend of mine told me that I should advertise my passion and know-how about SIMD more, and I decided to follow his advice. Though I am terrible at marketing and even more at personal marketing, I've made an attempt to do just that, advertise the fact that I'm offering SIMD Optimization Services (with emphasis on PowerPC AltiVec/VMX/VSX, and ARM NEON, but I'm ok with SSE as well, the logic is pretty much the same, though the difference(s) are in the details). For this reason I'm offering a free evaluation of your performance critical code (open/closed, able to sign NDAs if needed) to let you know if it's worth optimizing it, what kind of a performance gain you would get and how much it would cost you to get that result.
You can read more here.

22 September 2014

Konstantinos Margaritis: EfikaMX updated wheezy and jessie images available

A while ago, I promised to some people in powerdeveloper.org forum that I would provide bootable armhf images for wheezy but most importantly for jessie with an updated kernel. After a delay -I did have the images ready and working, but had to clean them up a bit- I decided to publish them here first. So, here are the images: http://freevec.org/files/efikamx-wheezy-armhf-20140921.img.xz (559MB)
http://freevec.org/files/efikamx-jessie-armhf-20140921.img.xz (635MB)

21 September 2014

Konstantinos Margaritis: VSX port added to Eigen!

Being the SIMD fanatic that I am, a few years ago I did the PowerPC Altivec and ARM NEON port for the Eigen linear algebra library, one of the best and most popular libraries -and most ported. Recently I thought it would be a good idea to extend both ports to 64-bit, and it would also help me with the SIMD book, using VSX in one case and ARMv8 NEON (or Advanced SIMD as ARM likes to call it) in the latter. ARMv8 hardware is a bit scarce at the moment, so I thought I'd start with VSX. Being in Debian, I have access to a number of porterboxes in several architectures, and luckily one of those was a Power7 (with VSX) running ppc64. So I started the porting -or rather extending the code- to use VSX in the 64-bit doubles case. Unluckily, I could not test anything because Debian kernels do not have VSX enabled in wheezy -which is what the porterbox is running and enabling it is a non-option(#758620). So, running VSX code would turn out to be quite hard.

4 August 2014

Konstantinos Margaritis: SIMD book, "Sponsored by ARM"!

Ok, took a while but I got the final word about this and can announce that the sponsor who donated 500 EUR to the Indiegogo campaign was ARM itself! I have to thank my friends at ARM@Cambridge and especially Dr Monika Biddulph, General Manager, Partner Enablement Group at ARM. When the book goes to print you can be sure it will include "Sponsored by ARM" somewhere! :) Also a friendly reminder that even if the campaign is over, I still welcome the support in the form of preorders/sponsorships.

30 July 2014

Konstantinos Margaritis: SIMD book, update 3! Addition/Subtraction mostly finished

Finally. Apologies for the delay, but it's been a busy month. This time I will hold true to my word and upload a PDF for people to see (attached to this page). So, what's new? Here is a list of things done: * Finished ALL addition/subtraction related instructions for all engines and major derivatives (SSE*/AVX, VMX/VSX, NEON/armv8 NEON). With diagrams (these were the reasons it has taken so long).
* Reorganized the structure (split the book into Parts I/II, the instruction index will be in Part II, Part I will carry the design analysis of each SIMD engine.
* Added an TOC/index.
* So far, with just Addition/Subtraction Chapter and the rest empty sections, it has reached 175 pages (B5, again I'm not fixed on the size, it might actually change)! My estimate is that the whole book will surpass 800 pages with everything included. TODO:

9 June 2014

Konstantinos Margaritis: SIMD book, second update!

From the Indiegogo page: https://www.indiegogo.com/projects/simd-engines-comparative-reference-guide/x/4966960#activity
  • Added titlepage (simple, but will do the job)
  • Reorganized ALL instructions to include both unsigned/signed in the same entity
  • Added Saturated Addition, Subtraction and Saturated Subtraction
  • Added ARMv8 NEON instructions taken from ARM infocenter draft
  • Fixed some instructions (added 64-bit arithmetic for NEON)
  • Added some special addition/subtraction, like add/sub with carry(vmx/vsx), addsub(SSE3/AVX)
  • Added some in-vector sum additions, sum reductions but no descriptions yet
  • Added diagram for 8-bit addition/subtraction (still need lots more).
  • Removed VMX128, couldn't find enough information, an email to some IBM toolchain developers was left unanswered, so I guess noone really will really care if that engine is left out, if enough people insist on it, please also be kind enough to provide some documentation on it.

3 June 2014

Konstantinos Margaritis: SIMD book, first draft published!

Check activity here: https://www.indiegogo.com/projects/simd-engines-comparative-reference-guide/x/4966960#activity From the update: Ok, I've been busy the past days, I started writing the book (using LaTeX :), and I'd like to say that progress has been good. I fixed the current list of SIMD engines that I'm going to include and it's a long one:

20 May 2014

Laura Arjona: Some experiences, and TODO, about fonts

TL;DR I don t know much about fonts, I just use the stock ones that come with my system. From time to time I have issues with docs that others create, and use other fonts. This post is about my plans on learning a bit more and, at least, know how to solve those issues, if it s possible, while staying in the bright side (the free-software / free font side). Long version The context I use Debian, LibreOffice, sometimes Inkscape, and LaTeX. One of my favorite hashtags is #iloveplaintext, I don t know much about design in general, and fonts and typography in particular. I didn t change the fonts in my desktops (only reduced the size in the laptop, to be able to read a bit more in my low resolution screen), I rarely change the font in Writer (maybe from Liberation Serif to Liberation Sans), I never changed the font in a LaTeX document or LaTeX beamer presentation (I write boring documents, I know) and when I paste from the web, it s usually because I want to read a long article so I paste as plain text in gedit and print it or save it. So I ve never felt to learn more about fonts, it just works, and covers my needs, (or not, but I mostly could live with the issues). Then a friend in the Pump.io network, Adri n Perales, published a blog post about Typography (in Spanish) that I liked very much, and I began to think (and remember) some of the issues that I have from time to time with fonts. Issue #1: League Gothic: a free font that was not installed in my system Today, again in the Pump.io network, I discover that the FSF published a poster Privacy is impossible without free software , in SVG format, but it didn t look well when I opened it with GIMP, clearly due to some missing font. Nice that SVG format is a plain text format (XML)! So I opened the file with gedit and searched for the text string whose font was missing. It was League Gothic font, of course a free software font, but not packaged in Debian, it seems. No problem. I downloaded the font, copied the files in /usr/share/fonts and problem solved. Issue #2: Book Antiqua: a non-free font (must find equivalent) A document made with Microsoft Word that one friend sends to me so I review and resend (in PDF format) to other people. It s a leaflet, and it has text in Arial, in Tahoma, and in Book Antiqua. When I open it with LibreOffice the aspect is wrong (the substitutes are not the same size so some breaking lines and so). Book Antiqua is not free. I learned that it s an imitation of the Palatino font, and that a similar font in the free software systems is URW Palladio . In Writer (LibreOffice), I went to Tools > Options > Fonts and declared the equivalence of the two fonts, so the program would use URW Palladio as a substitute of Book Antiqua. I opened again the document and it was quite better, very similar to the original aspect. I didn t bother in changing the text in Arial or Tahoma, since the substitutes that LibreOffice used were quite good. But I bookmarked this page: A Web Designer Guide to Linux Fonts for remembering the different fonts that I can try to emulate the Windows ones. I also know that I can install the Microsoft Core Fonts for the web since they are packaged for Debian in the contrib archive. But I ll try to survive without them for now (until now, I didn t bother, why should I now that I have the substituting guide?). In other news, I got impressed that MS Core Fonts is #4 in Most downloads all over the time in SourceForge, with more than 450 million downloads \o/ Issue #3: Installing a new free font in Debian So I decided to install one of the fonts that Adri n Perales recommended in his blog post, Linux Libertine . Since it is packaged for Debian, it s super easy: 
# apt-get install ttf-linux-libertine
(my LibreOffice was opened, so I closed it and opened again, and the font was there ready for use). Issue #4: Use a different font in LaTeX Well, as always, there is not one but many ways to do that in LaTeX. My intuition tells me that if there is a LaTeX package for the font that I want to use, it s probably a nice idea to just use it. So I searched about Linux Libertine in LaTeX and yes, there is a package (and you can find a very interesting font guide in The LaTeX font catalogue ). I installed the package texlive-fonts-extra, and then, I added two lines in my LaTeX document: 
\usepackage libertine 
\usepackage[T1] fontenc
Compiled, and the resultant PDF was using Libertine font instead of Computer Modern. TODO (and/or wishlist) When opening a document that uses a font not present in my system, I maybe wouldn t notice that a font is missing and I see a substitute (and maybe not the best one)!. It would be nice that the program tells the user This file uses the font X, and it seems it s not in your system. I ll use font Y as a substitute . I m not sure if there is a standard way to know which package contains a certain font. I use a web search engine to try to find out, and the websites that I linked in the article. I have to investigate and learn a bit more about free fonts equivalents to the ones that other people use, and fonts in general, so my documents are more beautiful and people gets interested to know about the tools that I use to produce them. Well, I ve written a long blog post (thanks if you read until here!), solved some issues, and try some things, but not even capturing a snapshot to show here! It seems that I m still lazy, forgive me I hope at least this #plaintext is useful for you :)
Filed under: My experiences and opinion, Uncategorized Tagged: Debian, English, Fonts, Free Software, LaTeX, libre software, LibreOffice, Moving into free software, pump.io

Next.

Previous.